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§ (54) Title: APPARATUS AND METHOD FOR DISPLAYING A VIDEO IMAGE 

<N 

Q (57) Abstract: A method of displaying a video image comprises receiving sequential image frames at a processor. Each image 
►5T frame is processed to obtain a kinoform. A programmable diffractive element such as an SUM represents the sequence of kinoforms 
allowing reproduction of the image using a suitable illumination beam. 
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Apparatus and Method for Displaying a Video Image 

The invention relates to an apparatus and method for displaying a video image, 
for example using holographic techniques. 

Various arrangements are known for displaying images using holography, for 
example using computer generated holograms (CGH). One such known 
arrangement is described in US6437919 in which an electronically addressable 
spatial light modulator (SLM) is addressed to provide a 3D holographic 
representation of an image. As a result when the SLM is illuminated by an 
appropriate illumination source the image is reconstructed in the replay field 
(RPF). 

A problem with existing projection displays such as holographic displays is 
that light efficiency is low, and 3D displays require a large number of pixels. 

Furthermore, holographically generated 2D video images are known to possess 
significant advantages over their conventionally projected counterparts, 
especially in terms of definition and efficiency. However, the computational 
complexity of the current hologram generation algorithms preclude their use in 
real-time applications. Furthermore, even if the existing algorithms were fast 
enough, the images generated by them are not of sufficient quality for use in a 
display application, 

The invention is set out in the claims. 

Because of the use of kinoforms programmable onto a programmable 
diffractive element such as an LCOS (Liquid crystal over silicon) device 
configured for phase only modulation, the efficiency of the display is 
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significantly increased compared to existing systems which rely on amplitude 
modulation of an illuminating beam. As a result real time video with minimal 
light loss may be achieved. In addition the invention may provide improved 
noise mitigation techniques by time multiplexing multiple iterations of 
5 kinoforms generated during an optimization algorithm. 

Furthermore, reconstructed kinoform images possess significant advantages 
over their conventionally projected counterparts, especially in terms of 
efficiency and robustness to pixel loss, in conjunction with a device to display 
10 the kinoform dynamically at video frame rates and hardware for computing the 
necessary phase distributions in real time. 

In the case of kinoforms electronically written onto an LCOS device, it will be 
appreciated that the term kinoform embraces any phase distribution 

15 representative of a phase hologram, where only the phase modulation of an 

object wavefront is calculated. For ease of expression the term "kinoform" and 
"hologram" are used interchangeably in the present description. Reference to 
video is not limited to data representing any particular type of image content 
and includes data that is representative of one or more still images, wherein the 

20 data are periodically or regularly refreshed. 

Embodiments of the invention will now be described with reference to the 
drawings, of which: 

25 Fig. 1 is a block diagram showing an architecture of a display according to the 
present invention; 

Fig. 2 shows schematically the components of a first embodiment of the 
invention; 
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Fig, 3 shows schematically the components of a second embodiment of the 
invention; 

Fig. 4 shows schematically the components of a third embodiment of the 
invention; 

5 Fig. 5 shows schematically the components of a fourth embodiment of the 
invention; 

Fig. 6 shows schematically the components of a fifth embodiment of the 
invention; 

Fig, 7 shows schematically the components of a sixth embodiment of the 
10 invention; 

Fig. 8 shows a partial cross-sectional view through an SLM; 

Figs. 9a and 9b are a respective view of an SLM showing alternative binary 

phase modulation schemes; 

Fig. 10 shows a block schematic drawing of processor components according 
15 to the invention; and 

Fig. 1 1 is a schematic drawing showing components of a seventh embodiment 
of the invention. 

In the following description like reference numerals refer to like parts between 
20 respective figures. 

In overview this application concerns an equipment for projecting video images 
from computer generated phase-only ho lograms^ (commonly known as a 
kinoforms). The kinoforms must be capable of being rapidly reconfigured, so 
25 they are presented on pixelated liquid crystal over silicon (LCOS) spatial light 
modulators that have been constructed to operate efficiently as phase-only 
modulators. The LCOS phase-only SLM is illuminated by a suitably expanded 
beam from a partially coherent light source (such as a laser or super 
luminescent light emitting diode, LED) and the phase-only kinoform is 
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converted to a real intensity image in the far field. The far field may be brought 
nearer using a projection lens to carry out this Fourier transformation. The 
video frames or sub-frames that make up the video images are converted into 
kinoforms by computing their Fourier transform at high speed by using 
5 hardware based fast processors (eg FFT, FPGA, and DS processors) 

and by (for example) using methods related to the Gerchberg-Saxton algorithm 
for phase retrieval (reference J.R. Fienup, "Phase retrieval algorithms: a 
comparison" Applied Optics vol 21 (15) pp. 2758-2765 (1982) and R.W. 
Gerchberg and W.O. Saxton "A practical algorithm for the determination of 
10 phase from image and diffraction plane figures" Optik 35 pp. 237-246 (1971 ), 
both of which are incorporated herein by reference. 

Referring to Fig. 1, a video display comprises a holographic video projector 
designated generally 10. An object 12 comprising a video image to be 

1 5 projected is received by a processor 14 and processed to form a kinoform on a 
programmable diffractive element 16 for example a spadal light modulator, 
The kinoform 16 is illuminated by an illumination source 18 to form an image 
20. Optics 22 are provided between the kinoform 16 and image 20 for 
projecting the replay on a screen as well as, for example for removing zero 

20 order effects or mitigating noise in the image. 

The kinoform to be generated in the processor 14 and programmed for 
representation by the SLM 16 comprises a representation of a phase hologram 
for example of the type described in L.B. Lesem, P.M. Hirsch, J.A. Jordan, 
25 "The kinoform: a new wavefront reconstruction device", IBM J. Res. Devel. 
pp. 150-5 (March, 1969). As this technique relies on phase only modulation 
there is no light loss through amplitude modulation. Furthermore the SLM is 
diffractive rather than imaging. The processing steps include generation of the 
kinoform, encoding of the kinoform into the SLM and noise mitigation steps as 
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described in more detail below. As a result a high quality, high spatial and 
temporal resolution projection of complex time varying images, or rapidly 
refreshing images, is provided. 

5 Figs. 2 to 5 show a single stage architecture according to first to fourth 

embodiments of the invention. Fig. 2 shows various of the components of the 
block diagram of Fig. 1 in more detail. For example the illuminating optics 
comprise a laser 200 together with a beam expander 202. The programmable 
diffractive element comprises an SLM 206 projecting onto a screen 208. An 

1 0 intermediate optical element comprising a polarising beam splitter 204 is 

provided in reflection/transmission mode, reflecting the illuminating beam onto 
the SLM 206 which in turn reflects via its mirror the modulated beam onto 
screen 208. It will be appreciated that the individual components can be any 
appropriate component and, for example, the laser, beam expander, and 

15 polarising beam splitter can be selected according to the optical requirements 
of the system. Conventional LCOS devices are suitable for the SLM when 
modified for phase-only modulation for example as supplied by Holoeye and 
CRLO Displays. Such modification may comprise binary phase using 
ferroelectric liquid crystal LCoS devices for phase modulation with ideal 

20 switching angle increasing from 45 degrees to 90 degrees for phase 

modulation. In that case no polarisers or analysers are required. Alternatively, 
high speed analogue LCoS devices can be introduced using liquid crystal 
electro-optic effects in which the optic axis rotates in the plane of the liquid 
crystal layer e.g. eiectroclinic effect, V-shape switching or the flexoelectric 

25 effect in nematic liquid crystals. In all these cases the ideal switching angle is 
also 45 degrees, which is approximated by integrating a quarter wave plate 
(QWP) over the mirrors of the LCoS device (not shown). The combination of 
the mirror and the QWP double the effective switching angle and enables 
electro-optic effects with switching angles near to 45 degrees to be used. An 
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alternative implementation is LCoS devices with an analogue response to the 
pixel, voltage using nematic liquid crystals, These differ from conventional 
nematic LCoS devices of this type that are used for intensity only projection. 
The configuration of the liquid crystal may be modified to give a better 
5 analogue phase response and the arrangement of the LCoS device relative to 
the polarisation state of the incoming device is changed. In that case, no 
analyser is required for phase-only projection. Manufacture of LCoS devices is 
further described in "The resurgence of LCOS displays" by H.C. Stauss in 
Information Display (November, 2004; vol. 20(1 1) pp. 16-20). 

10 

Referring to Figure 8, a cross-sectional view through a part of one embodiment 
of a binary SLM (100) is shown. The SLM (100) has plural pixels (120-123) in 
the drawing although it will be understood that the pixels are in fact arranged in 
a two dimensional matrix. Each pixel (120-123) is capable of being electrically 

15 controlled so as to provide a phase shift of 0 or n. Each pixel (120-123) of the 
SLM has a respective phase-mask pixel (130-133) overlying it and disposed in 
correspondence with it. As will be seen in the Figure the first, third and fourth 
phase mask pixels (130, 132, 133) have a relatively low thickness whereas the 
second phase mask pixel (131) has a relatively high thickness. The thickness 

20 difference is selected on the basis of the material of the phase mask and on the 
basis of the wavelength of the light used. The selection is such that the light 
passing through the thicker pixels (131) is subject to a phase shift of n more 
than the light passing through the thinner pixels (130,132,133). 

25 Where the SLM or like device is reflective, the light will pass twice through the 
phase mask, and corresponding changes in thickness will be needed. 

In the present embodiment the phase mask is formed as a coating on the SLM 
(100). It would of course be possible to use other arrangements for example use 
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two substantially identical SLMs with one providing a data display and the 
other forming the phase mask. 

5 A particular advantage of the arrangement shown in Fig. 2 is that, because of 
the use of phase distributions in the form of a kinoform, some or all of the 
projection optics can be incorporated into the kinoform on the LCOS device as 
a quadratic phase term for low-cost systems such as a micro-projector 
providing a Fresnel on- axis hologram. 

10 

Referring to Fig. 3, an alternative embodiment using a Fraunhofer (or Fourier) 
hologram is shown in which an additional lens 210 is incorporated providing 
projection optics. 

15 In order to remove the zero order spot in the hologram, which is a major noise 
source in the projection, if a linear phase term is added to the hologram then a 
Fresnel or Fraunhofer off-axis arrangement can be adopted as shown in Figs. 4 
and 5 respectively in which the screen 208 and, as appropriate, projection 
optics 210 are provided off-axis hence losing the zero order spot. 

20 

Referring to Figs. 6 and 7 two-stage structures are shown in fifth and sixth 
embodiments. According to these structures additional intermediate optics 
comprise an optically addressable spatial light modulator (OASLM) providing 
additional noise mitigation. 

25 

Referring firstly to Fig. 6, a kinoform is represented on an SLM 600 and is 
multiplexed onto an OASLM 604 via replication optics 602. As a result, 
multiple versions of the same or different phase hologram are spatially 
multiplexed on the OASLM, and these are projected via transform projection 
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optics onto a screen 608 according to a scheme used for amplitude holograms 
as described in US patent no. 6437919, the contents of which are fully 
incorporated herein by reference. 

5 The spot size of the projected image is inversely proportional to the aperture of 
the illumination beam and proportional to the effective focal length of the 
Fourier projection system. The small aperture provided by conventional SLMs 
such as LCOS ( 1 2-32 mm diagonal) is increased by the spatial multiplexing of 
the image onto a larger OASLM. As a result existing SLMs can be used 

1 0 without modification, because high quality modulated images are ideal for 
optically writing on the OASLM photo sensor in real time. Diffractive loss 
from the large number of pixels on the SLM is reduced by multiplexing the 
image onto contiguous tiles on the OASLM which hence effectively amplifies 
the first order diffracted by the SLM. The spot size of the projected image is 

15 made smaller by the resulting larger optical aperture of the system and in 
addition the structure of the OASLM is well suited to this implementation 
because of the absence of fine pixellation which would otherwise produce 
diffractive loss into higher orders and because the structure is appropriate for 
driving ferro-electric liquid crystals giving robust phase modulation as 

20 described in reference S. Mias, I.G. Manolis, N. Collings, T.D. Wilkinson, and 
W.A. Crossland, "Phase-modulating bistable optically addressed spatial light 
modulators using wide-switching-angle ferroelectric liquid crystal layer" 
Optical Engineering vol 44 (1) (2005) incorporated herein by reference. As a 
result a high contrast image is provided. 

25 

Fig. 7 shows an alternative two-stage structure in which multiple bit frames of 
SLM 610 provide multilevel holograms for spatial multiplexing on the 
OASLM for example respective decompositions of the kinoform by colour, 
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grey scale or otherwise as described in more detail below, which are then 
combined to provide a final image. 

Encoding Kinoforms: Binary and Multilevel Kinoforms. 

5 

It will be appreciated that the replication and transform projection optics can be 
of any appropriate type, for example the transform projection optics can be a 
Fourier Transform lens. Similarly the OASLM can be of any appropriate type, 
for example as described in S. Mias et al Opt Eng Jan 2005 incorporated herein 
by reference. 

It will be appreciated that any appropriate encoding scheme can be adopted, for 
example binary or multi-level quantisation. According to the binary scheme 
shown in Fig. 9a, an SLM 900 includes a plurality of pixels or switching 
elements 902 with a full switching angle 0 - 90°. As a result of the quarter 
wave thickness of the LCOS device, no zero order spot is formed in the far 
field 906. 

An alternative SLM comprising an LCOS device is shown in Fig. 9b in which 
the SLM 900 includes a switching element 902 having a full switching angle 
less than 90°. Accordingly as can be seen from the graph showing symmetrical 
distribution of E out 904, in the far field 906 a zero order spot is formed as a 
result of which, for example, the off-axis technique described with reference to 
Figs. 4 and 5 can be implemented. 

In an alternative encoding/quantisation scheme, further advantages can be 
achieved over binary phase devices (see below). According to this alternative 
embodiment multi-level quantisation is adopted in which case more of the light 
is directed to the desired image. Devices which provide multi-level 
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quantisation include zero twist nematic LCOS devices with positive dielectric 
anisotropy (E Hallstig, T Martin, L Sjoqvist, M Lindgren, "Polarisation 
properties of nematic liquid crystal spatial light modulators for phase 
modulation". Jul. Opt. Soc. Am. A., August 2004). Pi-cell nematic liquid 
5 devices might also be used, as might vertically aligned nematic liquid crystal 
devices and two frequency nematic devices ( K. Bauchert, S. Serati, A. 
Furman,"Dual frequency nematic SLMs for phase modulation", Proc SPIE 
4734 35-43 (2002)). Smectic liquid devices that may have faster switching 
speeds that may be used include electroclinic LCOS, deformed helix LCOS, 

10 tandem binary LCOS and V-shaped switching LCOS M. O'Callaghan, 

"Switching dynamics and surface forces in thresholdless V-shaped switching 
ferroelectric liquid crystals" Phys Rev. E67, 011710 (2003). 
The primary advantage of using multiple levels of phase is that the kinoform is 
capable of directing all the light into one diffracted order as opposed to the 

15 equal distribution of intensities into plus and minus symmetric orders in the 
case of binary kinoforms. Moreover, the kinoform can be represented with a 
reduced quantisation error resulting in lower noise in the replay field. Higher 
information content in the kinoform results in higher fidelity reconstruction. 

20 Computation of kinoforms in real time. 

The manner in which the apparatus described above can be utilised to produce 
real time video holographic projection will now be described. In a first method 
embodiment, referring to Figure 10, a signal (802) representing anmxm pixel 
25 input intensity field T xy , generated directly from a video signal is input to the 
processor (820) having a first processing block (801). The processing device 
(20) in this embodiment includes a field programmable gate array (FPGA) 
running code to enable it to perform the required functions. In other 
embodiments an ASIC is used and in yet others a programmed general purpose 
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computer is used. The first processing block (801) for each said plurality of 
pixellated holograms, forms a first data set at an output (803) by the 
expression = ^/r^expj j®™ ) » so that the first data set has amplitude 
equal to the amplitude of the desired pixel and identically distributed (i.i.d.) 
5 uniformly random phase. 

The first data set (803) is applied to a second processing block (804) which 
forms a second data set such that -F "'[j^'] at output (5) where F 3 
represents the 2D inverse Fourier transform. 

10 

The second data set is then shifted by third processing block (806) in the real 
direction in the complex plane (i.e. to the right) sufficiently to form a third data 
set (807) in which the phase of each data point is small. The third processing 
block (806) forms R as the smallest positive real such that \g^\< R \fx,y,n . 
15 and adds a real a where a» R to each data item of the second set to form a 
third set a+G™ at output (807) 

The third data set at output (807) is applied to a magnitude -forming fifth 
processing block (808) performing the function =\a+G^ ] \ such that 
20 is output as a fourth data set (809). 

The fourth data is then supplied to a binarisation stage (810) performing the 
f-1 if M w <0 M 

function = \ , f > * B) where Q w = median(Af £') to form a fifth 
data set (811) for display as a said hologram. 

25 

The fifth data set (11) is supplied to a ferroelectric liquid crystal SLM (12) for 
display and viewing. Although a ferroelectric liquid crystal spatial light 
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modulator is used in this embodiment other devices, for example including 
nematic SLMs, OASLMs (optically-addressed spatial light modulators), and 
also more exotic types of liquid crystal displays such as electroclinic, pi cell, 
flexoelectric, antiferroelectric, ferrielectric, V-shaped switching cells, and 
5 guest-host dye cells may be substituted, in addition to non-liquid crystal 
technologies such as MEMS devices such as DMDs. The device may be 
transmissive or reflective. 

The presence of only a single computationally intensive step , i.e. the inverse 
10 Fourier transform, allows current hardware to generate in real time plural, e.g. 
40, holograms per video frame. 

In a second method embodiment, a modified process generates sets of 2N 
distinct m x m binary phase holograms , each of which produces a replay 
15 field that approximates the same target image. The key feature of the process is 
that the noise fields produced by each hologram are i.i.d., satisfying the 
conditions above. 

The process begins with the specification of a target intensity image T xy and 
20 proceeds as follows : 

1 . Let = ^exp ( jQ™ ) where is uniformly distributed between 0 
and 2jt for l<n<N,l<x,y<m 

2. Let G^ , =F" 1 [r^ ,> ] where F" 1 represents the 2D inverse Fourier 
25 transform operator, for l<n<N 

3. Let = '&{G% ) } for 1 < n < N 

4. Let M [ ** N) = 3{ G ( £ } for 1 < n <, N 
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5. 



Let 




-1 ifM^<Q ln) 
1 if M™>Q ln) 



where Q {n) = medianCM^') 



and 



1 < n <; 2N 

Steps 3 and 5 of this process are exactly equivalent to steps 3, 4 and 5 of the 
5 first embodiment, but an additional step (number 4 in this process) is added. 
The holograms M { £ produced here are thus exactly the same as the holograms 
produced by the original algorithm, only here the holograms M^ +N) are 
also provided "for free" -i.e. without a need for further Fourier transform step. 

10 Step 1 forms N targets equal to the amplitude of the supplied intensity 
target T^, but with i.i.d. uniformly-random phase, Step 2 computes the N 
corresponding full complex Fourier transform holograms G™. Steps 3, 4 
compute the real part and imaginary part of the holograms, respectively. 
Binarisation of each of the real and imaginary parts of the holograms is then 

15 performed in step 5: thresholding around the median of ensures equal 
numbers of -1 and 1 points are present in the holograms, achieving DC balance 
(by definition) and also minimal reconstruction error. Hence, for one Fourier 
transform operation performed upon T xy , two binary phase holograms H xy are 
provided as opposed to one hologram obtained using the process of the first 

20 embodiment. 

Any purely real hologram (such as a binary phase hologram) produces a 
conjugate image in the replay field, reducing the usable area and hence the 
efficiency by half. 



25 



The conjugate image can be removed by generating a four-phase hologram 
where each pixel takes one of the values [1 although such holograms 
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cannot be displayed on an inherently binary device such as a ferroelectric SLM. 
More than two phase levels can be displayed on a nematic liquid crystal device, 
but these are currently orders of magnitude too slow for high frame-rate 
applications, as provided by the algorithm described above. 

5 

However by etching a pixellated phase mask of pixel values 1 and j (thus 
imposing phase shifts of 0 and n/2 respectively) on the top surface of a binary- 
phase SLM the conjugate image can be removed. This is achieved by 
randomly setting the pixel values of the mask such that the net modulation 

10 imposed by the combination of the SLM and phase mask lie in the set 

j], which provides the extra degree of freedom required to remove the 
conjugate image (at the expense of additional RPF noise), despite the fact that 
the SLM itself is binary. This technique has been used in correlator design and 
is here applied to the display application. Although the pixel values of the mask 

15 are random, and thus have "dc balance" in that there are statistically equal 
numbers of each value of phase shift, the mask itself is fixed and the pixel 
position of each value of phase shift is known. 

In other embodiments, a distribution of phase shifts that is not completely 
20 random is used. 

To modify the algorithm described above to use a phase mask, the second 
block (4) is modified to provide a such that 




25 where P xy defines the phase mask, which has been generated randomly so that 
each pixel has an equal probability of taking the value 1 or j. 
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To evaluate the efficacy of this technique in simulation, holograms were 
generated for the two test patterns with each hologram being replicated twice, 
the replication step being performed between steps 2 and 3 in the algorithm. 
The replay fields formed demonstrated that the conjugate image is removed 
5 effectively, and simulations also show that the use of the phase mask does not 
significantly degrade the signal-to-noise ratio of the RPF produced. 



The foregoing discussion relates to producing desired 2D structure in the far 
field using Fourier holograms. The method can also be extended to produce 
Fresnel holograms for the production of structure in the near field, which is 
particularly useful for a lensless optical configuration. As discussed above, 
Fresnel holography offers a further advantage of enabling lensless projection, 
since the hologram itself encodes the focussing element, giving an obvious 
weight, cost and size saving over conventional projective display systems. 

The discrete Fresnel transform is closely related to the Fourier transform, and 
describes the near field T xy produced at distance f by the hologram G xy (of 
dimensions X by Y and corresponding pixel sizes A x and A y ) when illuminated 
with coherent light of wavelength A.. When imaging not too close to the 
hologram the transform is given by 

T» = 
v Xf 



with corresponding inverse transform 



25 Hence if processing block (4) provides a such that 
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Fresnel holograms are generated that form the target image at the specified 
distance f from the hologram. 

5 

The techniques described may be modified for the generation of 3D holograms 
for 3D holographic video display. A 3D hologram of an object is simply a 
recording of the complex electromagnetic field (produced by light scattered by 
the object) at a plane in front of the object By Huygens' principle, if the EM 

10 field distribution on a plane P is known, Huygens wavelets can be propagated 
through space to evaluate the field at any point in 3D space. As such, the plane 
hologram encodes all the information necessary to view the object from any 
position and angle in front of the plane and hence is, in theory, optically 
indistinguishable from the object. In practice, limitations in the pixel resolution 

15 of the recording medium restricts the viewing angle 6 which, as in the 2D case, 
varies inversely with the pixel size A 

Consider a plane, perpendicular to the z-axis, intersecting the origin, and one 
point source emitter of wavelength X and amplitude A at position (X, Y, Z) 
behind it. The field F present at position (x, y) on the plane - i.e. the hologram 
20 - is given by 

F(x,y) = j~e?:p^^ withr = VU-X) 2 +(y-7) 2 +Z 2 

If a 3D scene is regarded as M sources of amplitude Aj at (X ( , Yj, Z,), the linear 
nature of EM propagation results in the total field hologram F being 

25 If F(x, y) is to be sampled over the region x^ a < x < , y^ a < y < y mi to form 
an m x m hologram F xy , there is obtained; 
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with r = M Xab + x Xmx ~ Xtti * -Xf + (y^ + y y ™*~ y .?*<. _V)» + z * 
V m m 

There is thus presented an algorithm (with an SLM phase mask) that generates 

N full-parallax 3D holograms for a given array of point sources, as above. 

Let = ^-^^^J^^ j + ~-j-r^ with r; as above where ^j" 3 is uniformly 

5 distributed between 0 and 2n for 1 < n <, N, 1 < i £ M 



pw 

Let - - JSL - , where P xy is the precomputed [l,j] phase mask as described in 
the previous section 

10 Let R be the smallest positive real such that |g^'|< R Vx,y,n . R exists since 

each value taken by G { £ is finite and so G™ has compact support 

Let - J^+G^j, where a is real and very much greater than R. 

< Q M , . 

/ . , , where £> ; = median(M„ ) 

15 To test this algorithm, there is considered the calculation of N = 8 holograms of 
resolution 512 x 512 and size 2mm x 2mm centred at the origin of plane P, 
giving a pixel size of A = 4um and hence a viewing angle of around 9 degrees 
under coherent red illumination (K= 632nm). The 3D scene used was a set of 
944 point sources that formed a wireframe cuboid of dimensions 12cm x 12cm 

20 x 18cm, located at a distance of 1.91m from the plane. 

The simulated RPFs produced were calculated by propagating Huygens 
wavelets from the N holograms in turn through a pinhole aperture K onto a 



Let ff« 
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virtual screen (a plane perpendicular to the line from the centre of the cube to 
the pinhole), and recording the time-averaged intensity distribution on the 
screen. 

5 Unfortunately, in contrast with algorithm described for 2D, the computation of 
step one in the 3D algorithm above may not be realisable in real time with 
current consumer hardware for a scene with anything but a very small number 
of point sources. However, the required holograms generated by the algorithm 
can be computed and stored for recall in real time, Much more powerful 
10 computation may be needed for real-time three-dimensional hologram 
generation, although it has been found possible to optimise the calculations in a 
way that suggests that such computation may not be needed. 

The holographic projection method detailed above is realisable in hardware. At 
15 the time of writing, commercially available FPGAs (field programmable gate 
arrays) can be programmed using commercially available code to compute 512 
x 512 Fourier transforms at a rate of at least 2400 frames/sec, sufficient for 
full-colour video (25 frames/sec, 3 colour planes, N = 32). For the display 
device, a commercially available SLM can display a sufficient frame rate at at 
20 least 512 x 512 resolution. It is possible to use separate illumination devices 
where multiple colours are needed -for example for a full colour display. 
Alternatively a single device capable of multiple colour output can be used 
instead. 

Two significant design issues have been identified. 

25 

Firstly, since optical loss in the phase holographic element is small and total 
energy is conserved, nearly all of the incident optical energy is routed into the 
RPF. Therefore, a target RPF with fewer "on" points will appear brighter than 
one with many, assuming total energy is distributed approximately evenly 
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between each "on" pixel in the RPF. As a result, a controller causes the 
intensity of the source to be modulated correspondingly to achieve a uniform 
overall brightness between frames, in accordance with the number of "on" 
points in each frame. Direct amplitude modulation of a laser is undesirable due 
5 to nonlinearity, and thus in an embodiment such amplitude modulation is 
replaced by pulse-width modulation within each 400 \xs pulse interval with a 
duty cycle proportional to the desired brightness to achieve the required 
average intensity. 



10 Secondly, a design issue comes from the relationship between RPF size and 
illumination wavelength. This is overcome in one embodiment by a lens system 
corrected for chromatic aberration at the three wavelengths of the sources and 
in another by using Fresnel holography to build compensation for this effect 
directly into the holograms. 

15 

The results of the approach of the invention show in simulation a RPF 
exhibiting a noise energy of two orders of magnitude lower than DBS, with 
calculation times six orders of magnitude faster. Experimental results are in 
agreement with the theory, and show a previously undemonstrated level of 
20 contrast and accuracy for a holographically generated image even with a test 
device that is a very old design of ferroelectric SLM. 

An alternative class of algorithms that can utilise the hard wired fast Fourier 
transform processors are directional iterative optimisation algorithms such as 

25 the Gerchberg Saxton algoridim. These are discussed in more detail below and 
belong to the class collectively known as ping-pong algorithms also including, 
for example, IFTA, input/output algorithms of the type known to the skilled 
person. In such algorithms the start point is a random phase distribution or 
other deterministic phase distribution, for example, some pre- determined phase 

30 distribution which is binarised and a Fast Fourier Transform (FFT) performed 
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thereon to give an improved phase distribution replacing the initial phase 
distribution. The cycle is reiterated until the EFT binarised phase distribution 
approximates to a desired target distribution. Alternatively, the random phase 
distribution generally assumed at the start of the GS algorithm can be replaced 
5 with a phase distribution inherited from a previous frame. 

This is particularly of interest when attempting to minimise the number of 
iterations necessary to achieve a good kinoform. When the video frame 
comprises a sequential series of sub-frames, this is also true of the sub-frames. 

10 

Alternatively algorithms such as uni-directional iterative optimisation 
algorithms (single pixel change and cost function testing) such as direct binary 
search and simulated annealing can be implemented, which techniques are once 
again well known to the skilled reader and are not described in detail here. 

15 

The kinoform can be generated in real time in a range of manners and 
alternative improved manners will now be described. As discussed above, it is 
desirable to generate holograms in M x real-time, where M represents the 
temporal multi-plexing factor for noise reduction and also for grey-scale/colour 
20 representation as discussed in more detail below. Algorithm implementation 
can be accelerated in a range of manners. 

In a first approach comprising hologram segmentation, an initial unoptimised 
hologram is sub-divided into equal areas, which are independently optimised 
25 according to an algorithm described herein in parallel. As the processing time 
is proportional to the square of the number of pixels in many instances, this 
correspondingly reduces the processing time distributed amongst parallel 
processors. In addition to spatial segmentation the hologram can be 
decomposed in any appropriate manner. 
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Yet further, quantisation procedures such as error-diffusion and projection onto 
constrained sets (POCS) may be applied after a complex hologram has been 
defined, again as well known to the skilled reader. In ail cases it will be 
5 appreciated that following segmentation or other decomposition of the 

hologram to optimise parallel processing, the decomposed parts are summed to 
give a fully reconstructed image. 

All these algorithms can be implemented and speeded up by using dedicated 
10 hardware in the processor for carrying out the optimisations, for example an 
FFT processor, FPGA (field programmable gate array) board, DCT processor 
(discrete cosine transform), or DSP (digital signal processing) board, of any 
appropriate known type. 

15 Modulation schemes and kinoform generation. 

It will be appreciated further that various possible modulation schemes can be 
implemented for representing the kinoform on the SLM, using binary or multi- 
level quantisation as discussed above. In particular modulation schemes can be 
adopted such that reconstruction of a hologram under sequential red, green, and 

20 blue illumination gives a coloured grey scale image. 

According to a direct modulation scheme, the SLM is directly modulated to 
provide colour and/or grey scale. In particular fast SLM devices are used to 
provide frame sequential multiplexing of binary or multi-level holograms to 

25 provide enhanced grey scale and frame sequential colour and, as discussed in 
more detail below, potential for noise reduction. Alternatively the illumination 
source can be modulated and the SLM can use holograms which replay to 
binary bit planes such that the light modulation dwell time or light intensity 
varies according to the significance of the bit plane, for example following a 

30 Frame Sequential Grey Scale (FSGS) scheme. Alternatively again, according 
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to a Frame Sequential Colour (FSC) scheme, the display can adopt a 
modulation scheme employing sequential illumination of red, green, blue 
(RGB) or other colour scheme light emitting diodes (LEDs) rather than a white 
light source in which case these are synchronised with holograms dedicated to 
5 a given colour. Schemes such as these are well known to the skilled reader and 
are not described in detail here. 



Additionally, it is desirable to reduce noise in the projected or displayed image, 
The sources of noise can be categorised as systematic and non-systematic. 

10 Systematic sources include phase representation errors and non-uniformity in 
the SLM. Non-systematic sources include noise arising from implementation 
of an optimisation algorithm, and binarisation errors. As discussed above, use 
of multi-level quantisation on the SLM reduces binarisation noise significantly 
and also reduces the noise introduced by optimisation algorithms as the 

15 constraints on the algorithm are reduced providing a great search space. Yet 
further multi-level quantisation reduces noise from phase representation errors. 
In addition noise averaging or noise cancellation techniques are employed 
where noise averaging spreads the effect of noise at the cost of a reduction in 
contrast whereas noise cancellation reduces noise without contrast reduction 

20 but with an increased processing burden. 

A first noise averaging approach comprises the algorithm described in more 
detail above in which a series of sub-frames with independent noise fields 
provide reduced noise. An alternative approach is to implement an iterative 
25 optimisation algorithm such as the above-mentioned Gerchberg Saxton 

algorithm taking some or all of the iterations as sub-frames. Each sub-frame 
displays statistical noise independence such that displaying the sub-frame 
sequentially provides noise reduction. The skilled reader will be fully familiar 
with implementation of the Gerchberg Saxton algorithm. In particular it will 
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be seen that each sub-frame is the result of a single iteration or "pass" of the 
algorithm based on a phase distribution generated by a previous pass. It will be 
seen that the algorithm can be used to generate sub-frames with each iteration 
being displayed between sub-frames to provide sub-frame-level noise 
5 reduction, or the iterations can be used to generate frames and displayed 
between frames in a similar manner. Sequential bit-planes and colour sub- 
frames may be used for the purpose of inheriting phase distributions, providing 
a better result than assuming a random phase distribution for each sub-frame. If 
sub-frames are used for other purposes (eg because they facilitate the 
10 computation of the holograms) then this principle might also be applied in this 
case. 

Alternatively again, noise averaging can be achieved by averaging over 
sequential decompositions of the total replay image, For example the image 
15 can be decomposed by colour, as discussed above, and multiplexed or summed 
to give the full image in which case there will be noise reduction. Alternative 
sub-frame sequential bit planes can be implemented or indeed any group of 
subsets of the full set of pixels can be used to give the total image, but 
sequentially represented to provide noise averaging. 

20 

Generally when projecting such sequential frames to produce a video image 
noise averaging occurs. This reduces the visible impact of any noise present in 
individual the frames. This is true whether the frames are generated from 
kinoforms or not, but it is particularly beneficial when they are (due to level of 
25 noise usually present within each frame in this case). It is even truer if the 
frames are composed of sequentially presented sub-frames since the noise is 
now averaged over both frames and sub-frames, and optimised results in 
achieving the best quality computer generated kinoform by minimising the 
amount of noise in image frames or sub-frames and maximising their efficiency 
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when projected, where multilevel quantised frames or sub-frames are 
generated. 

An example of the way in which this might be implemented for a first frame or 
subframe comprises the steps: 

5 

1) Assume random phase distribution over all the real pixels in the real image. 

2) Take the full Discrete Fast Fourier Transform (FFT). 

3) Reject the real part and take the FFT of the imaginary part. 

4) This gives a first iteration of the real image (which we will reject) along 
10 with an improved phase distribution to associate with it called PI . 

5) Take the FFT of real image plus the new imaginary part. 

6) Reject the real part. The imaginary part is now accepted as the phase 
hologram for frame 1 . 

15 For the next frame or subframe, this is likely to be very similar to the first. 

Hence the phase distribution PI is now associated with the next frame, and the 
FFT is taken using the real next frame and PL 

This can be continued for a number of frames on the assumption that it is likely 
20 that the frames are similar. 

At intervals the whole process is continuously repeated by returning to step 1 
(but in general it may not be necessary to use a random phase distribution when 
re-starting the sequence, as any inherited phase distribution may be just as 
25 good). 

One reason for repeating the sequence may be because eventually the scene 
will change and the inherited phase distribution will be no longer applicable, so 
it needs regenerating in steps 2 and 3. 
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Alternatively the process may be repeated back to step 1 more frequently than 
this so that the inherited phase distribution is continuously improved, even if 
the scene does not change substantially. Thus in general the process would be 
repeated back to step 1 as often as is possible , the limiting feature being the 
5 time required to carry out the Fast Fourier Transforms. 

This is particularly of interest when attempting to minimise the number of 
iterations necessary to achieve a good kinoform. The same principle might be 
applied to some kinds of sub-frames that might be projected in sequence to 

10 make up each full video frame. Examples of such sub-frames might include 
red, green and blue sub-frames in frame sequential colour projection systems 
(which are integrated by the eye to give a full colour image). They may also 
include the (binary) bit planes that may be projected in sequence to achieve a 
grey-level image (e.g. for each colour). In this case appropriate binary 

15 weighting is provided to each bit plane on projection by changing the intensity 
of the projection light source. 

The kinoform calculated as above or otherwise will be improved if the if 
multiple periodic copies of the kinoform are illuminated by the projector laser 
20 light source rather than just one copy. 

In the case of noise cancellation, the hologram generated can be of higher 
resolution than the target image. This can result in a quasi-periodic pattern 
being introduced as a result of the kinoform optimisation algorithm, each 
25 pattern having slightly different features to its neighbours, giving rise to noise 
reduction. 

A further alternative noise cancellation technique comprises spatial 
multiplexing of a hologram or a set of holograms on the SLM, or multiplexing 
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a single hologram spatially onto an OASLM. This once again provides more 
pixels in the kinoform which enhances replay quality. One such approach is 
described in L.B. Lesem, P.M. Hirsch, J.A. Jordan, "Computer synthesis of 
holograms for 3-D display", Commun. ACM, 11, 661-674 (1968). For 
5 example it will be appreciated that the sub-frames generated as described 
above, can be spatially multiplexed, i.e. the respective sub-frames can be 
juxtaposed on an OASLM and then be recombined as a single frame providing 
yet further enhanced noise reduction and image quality. 

10 Yet a further approach particularly applicable to systematic sources of noise 
comprises appropriate shaping of the beam wavefront from the illumination 
source to compensate for known systematic errors. Those errors can be 
identified in a calibration stage and appropriate waveform shaping optics 
implemented accordingly. 

15 

It will be appreciated that the individual components of the embodiments 
described above can be any appropriate optical or other elements. For example 
the processing device includes a field programmable gate array (FPGA) 
running code to enable it to perform the required functions. In other 
20 embodiments an ASIC is used and in yet others a programmed general purpose 
computer is used. The LCOS can comprise a modulatable pixel array driven by 
a separate processor. In an alternative configuration, the processing hardware 
can be incorporated into the LCOS back plane providing a fully intergrated 
device simply requiring a video input. 

25 

As a result of the embodiments described above, a fast, reduced noise, real- 
time video holographic projector with highlight efficiency and image quality is 
provided. 
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According to yet a further embodiment of the invention lossless light projection 
is achieved using a phase-only image as shown in Fig. 11. A collimated light 
beam 1102 illuminates a phase-only SLM 1104 which shows a phase 
modulated, direct (non-holographic) representation of the image. The phase 
5 modulated output beam is focused by optics 1 106 onto a phase contrast filter 
1 108 at the Fourier plane which is then focused via further optics 1110 onto an 
image plane 1 1 12. As a result, if a phase image of the object is presented on 
the SLM, then projection via the 4f imaging system shown provides an 
amplitude modulated image with high throughput efficiency. Any appropriate 
10 spectrally sharp, narrow band illumination can be provided and as coherency is 
not required, alternative light sources to a laser may be used, which reduces 
noise arising from speckle. In the preferred embodiment, multi-level 
modulation is implemented to provide a grey scale image. 

15 It will be appreciated that the embodiments described above can be used in any 
display or projection application including rear and front projection, and in 
relation to any regularly refreshed image in particular where real-time 
processing of a corresponding sequence of kino forms is desired. 

20 It will be appreciated that in the above description, the use of the term "pixel" 
covers any appropriate modulatable and addressable element of any shape or 
distribution. It is not intended herein, when the term "pixel" is used, that any 
directly viewable picture content is to be seen on the elements concerned. 

25 It will be appreciated that the invention extends to a computer program product 
implementing the processing steps, any computer or processor implementing 
those steps and any computer readable medium storing or otherwise capable of 
providing the computer program, for example as a media storage device such 
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as a CD-ROM, or in wired or wireless downloadable form from a remote server 
computer. 
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CLAIMS 

1. A method of displaying a video image comprising the steps of receiving 
sequential image frames at a processor, processing each image frame to obtain 

5 one or more respective kinoforms and programming a programmable 
diffractive element to sequentially represent each said kinoform. 

2. A method as claimed in claim 1 in which the video image is one of a 
time varying or still image. 

10 

3. A method as claimed in claim 1 or claim 2 in which the image is at least 
one of projected or directly viewed. 

4. A method as claimed in any preceding claim in which the image frames 
15 are processed in real time, 

5. A method as claimed in any preceding claim further comprising parallel 
processing respective sub-sets of an initial kinoform to obtain optimised sub- 
sets and recombining the optimised sub-sets. 

20 

6. A method as claimed in claim 5 in which the initial kinoform sub-sets 
comprise initial kinoform segments. 

7. A method as claimed in claim 5 or claim 6 in which the initial kinoform is 
25 randomly generated or pre-determined or the result of an initial FFT, or a 

recycled phase distribution. 

8. A method as claimed in any preceding claim in which image frames are 
processed to obtain a respective kinoform using any one of a bi-directional 
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iterative optimisation algorithm, a uni-directional iterative optimisation 
algorithm or a bi-directional iterative quantisation .procedure, or a uni- 
directional non-iterative procedure. 

5 9. A method as claimed in any preceding claim further comprising 
controlling' a parameter of operation to provide at least one of grey scale and 
colour. 

10. A method as claimed in claim 9 in which the programmable diffractive 
10 element is directly modulated to provide colour and/or grey scale. 

1 i . A method as claimed in claim 10 in which the programmable diffractive 
element provides a binary quantisation scheme and is multiplexed frame 
sequentially to provide colour and/or grey scale. 

15 

12. A method as claimed in claim 10 in which the programmable diffractive 
element provides a multi-level quantisation scheme to provide colour and/or 
grey scale. 

20 13. A method as claimed in claim 9 in which a kinoform illumination source 
is modulated in the time domain to provide colour and/or grey scale. 

14. A method as claimed in claim 9 in which a kinoform illumination source 
is modulated in the colour domain to provide colour and/or grey scale. 

25 

15. A method as claimed in any preceding claim in which the kinoform 
obtained comprises one of a plurality of kinoforms which each reconstruct a 
spatial or intensity decomposition of the image frame. 
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16. A method as claimed in claim 15 in which the plurality of kinoforms 
sum to give the real image. 

17. A method as claimed in any preceding claim further comprising a noise 
5 reduction step. 

18. A method as claimed in claim 17 in which noise reduction is provided 
by programming the programmable diffractive element to represent the 
kinoform according to a multi-level quantisation scheme. 

10 

19. As method as claimed in claim 17 in which noise reduction is provided 
by noise averaging over a plurality of image frames. 

20. A method as claimed in claim 17 in which noise reduction is provided 
15 by noise averaging over a plurality of sub-frames of each image frame. 

21. A method as claimed in claim 20 in which each sub-frame is generated 
in an optimisation algorithm. 

20 22. A method as claimed in claim 21 in which the optimisation algorithm is 
the Gerchberg Saxton algorithm. 

23. A method as claimed in claim 17 in which the image frame and 
kinoform are pixellated, and wherein the kinoform contains excess pixels in 

25 relation to the image frame. 

24. A method as claimed in claim 17 in which noise reduction is provided 
by spatial multiplexing of each phase hologram. 
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25. A method as claimed in claim 24 in which the phase holograms are 
spatially multiplexed on the programmable diffractive element. 

26, A method as claimed in claim 24 in which the phase holograms are 
5 spatially multiplexed on an optically addressable spatial light modulator. 

27. A method as claimed in any of claims 20 to 23 in which each image 
frame comprises a plurality of sequential image sub-frames and the spatial 
multiplexing step comprises juxtaposing sequential sub-frames. 

10 

28 A method as claimed in claim 17 in which noise reduction is provided by 
shaping an illumination source wave front to mitigate systematic noise. 

29. A method as claimed in any preceding claim further comprising spatially 
15 multiplexing the kinoform on one of the programmable diffractive elements or 

an optically addressable spatial light modulator. 

30. A video display element comprising an image processor and a 
programmable diffractive element, in which the image processor is arranged to 

20 receive and process sequential image frames to obtain respective kinoform and 
program the programmable diffractive element to represent said kinoforms 
sequentially. 

31. A display element as claimed in claim 30 in which the video image 
25 comprises at least one of a time varying and still image. 

32. A display element as claimed in claim 30 or claim 31 in which the 
programmable diffractive element comprises a spatial light modulator. 
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33. A display element as claimed in any of claim 30 to 32 in which the 
programmable diffractive element provides binary quantisation. 

34. A display element as claimed in claim 33 in which the programmable 
5 diffractive element includes one or more modulating elements. 

35. A display element as claimed in any of claims 30 to 32 in which the 
programmable diffractive element provides multi-level quantisation. 

10 36. A display element as claimed in any of claims 30 to 35 in which the 
programmable diffractive element includes, as a programmable component, 
any one of nematic liquid crystal on silicon (LCOS), electroclirdc LCOS, 
deformed helix LCOS and V-shaped switching LCOS. 

15 37. A display element as claimed in any of claims 30 to 36 further 
comprising imaging optics associated with the programmable diffractive 
element. 

38. A display element as claimed in any of claim 30 to 36 in which imaging 
20 optics are programmed into the kinoform. 

39. A display element as claimed in any of claims 30 to 38 in which the 
processor includes at least one of an FFT processor, a field programmable gate 
array processor or a digital signal processor. 

25 

40. A video display including a display element as claimed in any of claims 
30 to 39 and a partially coherent illumination source. 

41. A display as claimed in claim 40 in which the illumination source is 
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arranged to be modulated to provide colour and/or grey scale. 

42. A display as claimed in claim 40 or claim 41 further comprising 
illumination source wave-front shaping optics arranged to shape an 

5 illumination beam to mitigate systematic noise in the display. 

43. A display as claimed in any of claims 40 to 42 further including an 
optically addressable spatial light modulator (OASLM). 

10 44. A display as claimed in claim 43 further comprising spatial multiplexing 
optics arranged to spatially multiplex the kinoform onto the OASLM. 

45. A display as claimed in claim 43 in which the spatial modulation optics 
are further arranged to spatially multiplex temporally varying kinoform. 

15 

46. A display as claimed in any of claims 40 to 45 further comprising a 
display screen. 

47. A display as claimed in claim 46 in which the display screen is located 
20 optically off-axis. 

48. A processor for a video display element for a display as claimed in any 
of claims 30 to 47, arranged to receive and process sequential image frames 
according to a method as claimed in any of claims 1 to 30. 

25 

49. A computer program comprising a sequence of steps arranged to 
implement operation of a processor as claimed in claim 48. 

50. A computer readable medium configured to store a computer program 
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as claimed in claim 49, 

51 . A method of displaying a video image comprising the steps of receiving 
sequential image frames at a processor, processing each image frame to obtain 
5 a respective phase representation and applying said phase representation to a 
spatial light modulator. 

52. A method as claimed in claim 51 in which the phase representation 
comprises a kinoform. 

10 

53. A method as claimed in claim 51 in which the phase representation 
comprises an image representation, further comprising the steps of passing 
the represented image through a phase contrast filter. 

15 54. A video display element comprising an image processor and a 
programmable diffractive element in which the image processor is arranged to 
receive and process sequential image frames to obtain respective phase 
representations and programme the programmable diffractive element to 
represent said phase representations sequentially, 

20 

55. A display element as claimed in claim 54 further comprising a phase 
contrast filter. 

56. A method of displaying an image comprising the steps of receiving an 
25 image frame at a processor, generating a kinoform according to an iterative 

algorithm and displaying each iteration of the kinoform. 

57. A method as claimed in claim 56 in which the algorithm is the Gerchberg 
Saxton algorithm. 
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58. A method as claimed in claim 56 or 57 in which the image is a video 
image. 

5 59. A method of processing an image frame to obtain one or more kinoforms 
for a display or display method according to any preceding claim. 

60. A method, a display element or display substantially as herein described 
with reference to the drawings. 
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